Search Results: "francois"

19 November 2013

Francois Marier: Things that work well with Tor

Tor is a proxy server which allows its users to hide their IP address from the websites they connect to. In order to provide this level of anonymity however, it introduces latency into these connections, an unfortunate performance-privacy trade-off which means that few users choose to do all of their browsing through Tor. Here are a few things that I have found work quite well through Tor. If there are any other interesting use cases I've missed, please leave a comment!

Tor setup There are already great docs on how to install and configure the Tor server and the only thing I would add is that I've found that having a Polipo proxy around is quite useful for those applications that support HTTP proxies but not SOCKS proxies. On Debian, it's just a matter of installing the polipo package and then configuring it as it used to be recommended by the Tor project.

RSS feeds The whole idea behind RSS feeds is that articles are downloaded in batch ahead of time. In other words, latency doesn't matter. I use akregator to read blogs and the way to make it fetch articles over Tor is to change the KDE-wide proxy server using systemsettings and setting a manual proxy of localhost on port 8008 (i.e. the local instance of Polipo). Similarly, I use podget to automatically fetch podcasts through this cron job in /etc/cron.d/podget-francois:
0 12 * * 1-5 francois   http_proxy=http://localhost:8008/ https_proxy=http://localhost:8008/ nocache nice ionice -n7 /usr/bin/podget -s
Prior to that, I was using hpodder and had the following in ~/.hpodder/curlrc:
proxy=socks4a://localhost:9050

GnuPG For those of us using the GNU Privacy Guard to exchange encrypted emails, keeping our public keyring up to date is important since it's the only way to ensure that revoked keys are taken into account. The script I use for this runs once a day and has the unfortunate side effect of revealing the contents of my address book to the keyserver I use. Therefore, I figured that I should at least hide my IP address by putting the following in ~/.gnupg/gpg.conf:
keyserver-options http-proxy=http://127.0.0.1:8008
However, that tends to makes key submission fail and so I created a key submission alias in my ~/.bashrc which avoids sending keys through Tor:
alias gpgsendkeys='gpg --send-keys --keyserver-options http-proxy=""'

Instant messaging Communication via XMPP is another use case that's not affected much by a bit of extra latency. To get Pidgin to talk to an XMPP server over Tor, simply open "Tools Preferences" and set a SOCKS5 (not Tor/Privacy) proxy of localhost on port 9050.

GMail Finally, I found that since I am running GMail in a separate browser profile, I can take advantage of GMail's excellent caching and preloading and run the whole thing over Tor by setting that entire browser profile to run its traffic through the Tor SOCKS proxy on port 9050.

25 October 2013

Enrico Zini: A vision wanted

A vision wanted Today Richard Stallman mailed all Italian LUGs asking that tomorrow's LinuxDay be called "GNU/Linux Day" instead. I wonder how that is ever going to help a community so balkanised, that the only way Italian LUGs manage to do something together, is to say "let's not care what we all do, let's just do it on the same day and call it a national event". Of course a few LUGs still make a point of not doing anything on that day, because you know, Judean People's Front. Cawk. Today a friend asked me if I could help her support people in installing Citrix Whatsit to set up a video conference to network meetings that will take place in a month in different cities. Those meetings are something I look forwad to. It wasn't much of a problem to say "no, I can't do that"; it was a problem to be unable to come up with some viable, Free alternatives. I sometimes have to use Skype to talk with friends who also are Debian Developers, because I still haven't managed to make VoIP work unless I go through a commercial proxy. There was the happy news that our regional administration is switching from MS Word to OpenOffice. It soon became a flamewar, because some people loudly complained that they should have used LibreOffice instead. At DebConf, after spending an hour getting frustrated with the default formatting of bullet points in WhateverOffice Impress, I did my Debian Contributors talk using a text file in vim. And it was a success! Thanks Francois Marier for maintaining cowsay. I can't sync contact lists and appointments between my N900, which runs a Debian derivative, and my laptop, because I don't want to have a Google account, and nothing else would work out of the box. I don't even know how to keep a shared calendar with other DDs, without using a 3rd party cloud service that I don't want to trust with my life's personal schedule. I need to do a code review of every vim plugin I need to use, because you can only get them by cloning GitHub repositories over plain http, and they don't even provide signed tags. That code is run with my own privileges every time I start a text editor, which is, like, all the time. I'm frightened at the idea of how many people blissfully don't think about what that means. Vim users. Programmers. Cool chaps. Yet the most important thing in Debian today seems to be yet another crusade between upstart and systemd. But we haven't had a lengthy discussion on why, although the excellent OpenStreetMap exists and many of us contribute to it, it seems to still be more immediate to hit Google Maps to get a route computed. How can we change that? We haven't had a lengthy discussion on what can we offer to allow anyone to set up some social platform that won't get swamped with spam the first day and cracked open the second; that would allow people to share some photos with their friends only, and some with the rest of the world; that won't required a full-time paid person to maintain. That won't be obsolete and require a migration to a new platform in a year. That isn't Facebook or Google Plus. I stopped taking photos because it's too much work to show them to people. Other people use Instagram. Whatever the hipster trend is for photo sharing today, October 25, 2013, I'm pretty sure it's not a Free platform. But we can do something. We technology leaders. We are those who drive technological change! For example, today I invested two hours of hard effort trying to figure out why libbuffy's test suite fails on kfreebsd. All while wondering why I was doing that, since I know all buffy's users personally, and none of them uses kfreebsd. And I will take a day off work to study the library symbols file specification, so that next time I'll know right away if the new version of a C++ compiler decides that a template-generated symbol isn't worth adding to a library anymore. What is this effort really about? It sometimes feel like micromanaging to me. It's good to have excellent quality standards. But not without a vision. Not until "reliable network printing with all PDF viewers and print servers we ship" is among our release goals. Not until we commit to making sure that "sharing files between Debian users" will work out of the box, without the need of going through a 3rd party website, or email. I'm not interested in spending energy discussing init systems. I'm interested in spending energy sharing stories of what cool stuff we can do in Debian today, out of the box. And what cool stuff we'll be able to do tomorrow. Let's spend time on IRC, on mailing lists, and at the next Debian events, talking about why we are really into this. Talking about a vision! Note: Please don't spend time telling me how to fix the problems I mentioned above. I'm not interested in help fixing some problems for me today. I'm interested in asking help fixing problems for everybody, right in the next stable release. Remember, remember, the 5th of November, 2014.

15 October 2013

Francois Marier: The Perils of RAID and Full Disk Encryption on Ubuntu 12.04

I've been using disk encryption (via LUKS and cryptsetup) on Debian and Ubuntu for quite some time and it has worked well for me. However, while setting up full disk encryption for a new computer on a RAID1 partition, I discovered that there are a few major problems with RAID on Ubuntu.

My Setup: RAID and LUKS Since I was setting up a new machine on Ubuntu 12.04 LTS (Precise Pangolin), I used the alternate CD (I burned ubuntu-12.04.3-alternate-amd64+mac.iso to a blank DVD) to get access to the full disk encryption options. First, I created a RAID1 array to mirror the data on the two hard disks. Then, I used the partition manager built into the installer to setup an unencrypted boot partition (/dev/md0 mounted as /boot) and an encrypted root partition (/dev/md1 mounted as /) on the RAID1 array. While I had done full disk encryption and mirrored drives before, I had never done them at the same time on Ubuntu or Debian.

The problem: cannot boot an encrypted degraded RAID After setting up the RAID, I decided to test it by booting from each drive with the other one unplugged. The first step was to ensure that the system is configured (via dpkg-reconfigure mdadm) to boot in "degraded mode". When I rebooted with a single disk though, I received a evms_activate is not available error message instead of the usual cryptsetup password prompt. The exact problem I ran into is best described in this comment (see this bug for context). It turns out that booting degraded RAID arrays has been plagued with several problems.

My solution: an extra initramfs boot script to start the RAID array The underlying problem is that the RAID1 array is not started automatically when it's missing a disk and so cryptsetup cannot find the UUID of the drive to decrypt (as configured in /etc/crypttab). My fix, based on a script I was lucky enough to stumble on, lives in /etc/initramfs-tools/scripts/local-top/cryptraid:
#!/bin/sh
PREREQ="mdadm"
prereqs()
 
     echo "$PREREQ"
 
case $1 in
prereqs)
     prereqs
     exit 0
     ;;
esac
cat /proc/mdstat
mdadm --run /dev/md1
cat /proc/mdstat
After creating that file, remember to:
  1. make the script executable (using chmod a+x) and
  2. regenerate the initramfs (using dpkg-reconfigure linux-image-KERNELVERSION).
To make sure that the script is doing the right thing:
  1. press "Shift" while booting to bring up the Grub menu
  2. then press "e" to edit the default boot line
  3. remove the "quiet" and "splash" options from the kernel arguments
  4. press F10 to boot with maximum console output
You should see the RAID array stopped (look for the output of the first cat /proc/mdstat call) and then you should see output from a running degraded RAID array.

Backing up the old initramfs If you want to be extra safe while testing this new initramfs, make sure you only reconfigure one kernel at a time (no update-initramfs -u -k all) and make a copy of the initramfs before you reconfigure the kernel:
cp /boot/initrd.img-KERNELVERSION-generic /boot/initrd.img-KERNELVERSION-generic.original
Then if you run into problems, you can go into the Grub menu, edit the default boot option and make it load the .original initramfs.

18 September 2013

Francois Marier: Presenting from a separate user account

While I suspect that professional speakers have separate presentation laptops that they use only to give talks, I don't do this often enough to justify the hassle and cost of a separate machine. However, I do use a separate user account to present from. It allows me to focus on my presentation and not stress out about running into configuration problems or exposing private information. But mostly, I think it's about removing anything that could be distracting for the audience. The great thing of having a separate user for this is that you can do whatever you want in your normal account and still know that the other account is ready to go and configured for presenting on a big screen.

Basics The user account I use when giving talks is called presenter and it has the same password as my main user account, just to keep things simple. However, it doesn't need to belong to any of the UNIX groups that my main user account belongs to. In terms of configuration, it looks like this:
  • power management and screen saver are turned off
  • all sound effects are turned off
  • window manager set to the default one (as opposed to a weird tiling one)
  • desktop notifications are turned off
Of course, this user account only has the software I need while presenting. You won't find an instant messaging, IRC or Twitter client running on there. The desktop only has the icons I need for the presentation: slides and backup videos (in case the network is down and/or prevents me from doing a live demo).

Web browsers While I usually have my presentations in PDF format (for maximum compatibility, you never know when you'll have to borrow someone else's laptop), I use web browsers (different ones to show that my demos work with all of them) all the time for demos. Each browser:
  • clears everything (cookies, history, cache, etc.) at the end of the session
  • has the site I want to demo as the homepage
  • only contains add-ons or extensions I need for the demos
  • has the minimal number of toolbars and doesn't have any bookmarks
  • has search suggestions turned off
  • never asks to remember passwords

Terminal and editors Some of my demos may feature coding demos and running scripts, which is why I have my terminal and editor set to:
  • a large font size
  • a color scheme with lots of contrast
It's all about making sure that the audience can see everything and follow along easily. Also, if you have the sl package installed system-wide, you'll probably want to put the following in your ~/.bashrc:
alias sl="ls"
alias LS="ls"

Rehearsal It's very important to rehearse the whole presentation using this account to make sure that you have everything you need and that you are comfortable with the configuration (for example, the large font size). If you have access to a projector or a digital TV, try connecting your laptop to it. This will ensure that you know how to change the resolution of the external monitor and to turn mirroring ON and OFF ahead of time (especially important if you never use the default window manager or desktop environment). I keep a shortcut to the display settings in the sidebar.

Localization Another thing I like to do is to set my operating system and browser locales to the one where I am giving a talk, assuming that it is a western language I can understand to some extent. It probably doesn't make a big difference, but I think it's a nice touch and a few people have commented on this in the past. My theory is that it might be less distracting to the audience if they are shown the browser UI and menus they see every day. I'd love to have other people's thoughts on this point though. Also, pay attention to the timezone since it could be another source of distraction as audience members try to guess what timezone your computer is set to.

Anything else? If you also use a separate account for your presentations, I'd be curious to know whether there are other things you've customised. If there's anything I missed, please leave a comment!

8 August 2013

Francois Marier: Server Migration Plan

I recently had to migrate the main Libravatar server to a new virtual machine. In order to minimize risk and downtime, I decided to write a migration plan ahead of time. I am sharing this plan here in case it gives any ideas to others who have to go through a similar process.

Prepare DNS
  • Change the TTL on the DNS entry for libravatar.org to 3600 seconds.
  • Remove the mirrors I don't control from the DNS load balancer (cdn and seccdn).
  • Remove the main server from cdn and seccdn in DNS.

Preparing the new server
  • Setup the new server.
  • Copy the database from the old site and restore it.
  • Copy /var/lib/libravatar from the old site.
  • Hack my local /etc/hosts file to point to the new server's IP address:
    xxx.xxx.xxx.xxx www.libravatar.org stats.libravatar.org cdn.libravatar.org
    
  • Test all functionality on the new site.

Preparing the old server
  • Prepare a static "under migration" Apache config in /etc/apache2/sites-enables.static/:
    <VirtualHost *:80>
        RewriteEngine On
        RewriteRule ^ https://www.libravatar.org [redirect=301,last]
    </VirtualHost>
    <VirtualHost *:443>
        SSLEngine on
        SSLProtocol TLSv1
        SSLHonorCipherOrder On
        SSLCipherSuite RC4-SHA:HIGH:!kEDH
        SSLCertificateFile /etc/libravatar/www.crt
        SSLCertificateKeyFile /etc/libravatar/www.pem
        SSLCertificateChainFile /etc/libravatar/www-chain.pem
        RewriteEngine On
        RewriteRule ^ /var/www/migration.html [last]
        <Directory /var/www>
            Allow from all
            Options -Indexes
        </Directory>
    </VirtualHost>
    
  • Put this static file in /var/www/migration.html:
    <html>
    <body>
    <p>We are migrating to a new server. See you soon!</p>
    <p>- <a href="http://identi.ca/libravatar">@libravatar</a></p>
    </body>
    </html>
    
  • Enable the rewrite module:
    a2enmod rewrite
    
  • Prepare an Apache config proxying to the new server in /etc/apache2/sites-enabled.proxy/:
    <VirtualHost *:80>
        RewriteEngine On
        RewriteRule ^ https://www.libravatar.org [redirect=301,last]
    </VirtualHost>
    <VirtualHost *:443>
        SSLEngine on
        SSLProtocol TLSv1
        SSLHonorCipherOrder On
        SSLCipherSuite RC4-SHA:HIGH:!kEDH
        SSLCertificateFile /etc/libravatar/www.crt
        SSLCertificateKeyFile /etc/libravatar/www.pem
        SSLCertificateChainFile /etc/libravatar/www-chain.pem
        SSLProxyEngine on
        ProxyPass / https://www.libravatar.org/
        ProxyPassReverse / https://www.libravatar.org/
    </VirtualHost>
    
  • Enable the proxy-related modules for Apache:
    a2enmod proxy
    a2enmod proxy_connect
    a2enmod proxy_http
    

Migrating servers
  • Tweet and dent about the upcoming migration.
  • Enable the static file config on the old server (disabling the Django app).
  • Copy the database from the old server and restore it on the new server.
  • Copy /var/lib/libravatar from the old server to the new one.

Disable mirror sync
  • Log into each mirror and comment out the sync cron jobs in /etc/cron.d/libravatar-slave.
  • Make sure mirrors are no longer able to connect to the old server by deleting /var/lib/libravatar/master/.ssh/authorized_keys on the old server.

Testing the main site
  • Hack my local /etc/hosts file to point to the new server's IP address:
    xxx.xxx.xxx.xxx www.libravatar.org stats.libravatar.org cdn.libravatar.org
    
  • Test all functionality on the new site.
  • If testing is successful, update DNS to point to the new server with a short TTL (in case we need to revert).
  • Enable the proxy config on the old server.
  • Hack my local /etc/hosts file to point to the old server's IP address.
  • Test basic functionality going through the proxy.
  • Remove local /etc/hosts/ hacks.

Re-enable mirror sync
  • Build a new libravatar-slave package with an updated known_hosts file for the new server.
  • Log into each server I control and update that package.
  • Test the connection to the master (hacking /etc/hosts on the mirror if needed):
    sudo -u libravatar-slave ssh libravatar-master@0.cdn.libravatar.org
    
  • Uncomment the sync cron jobs in /etc/cron.d/libravatar-slave.
  • An hour later, make sure that new images are copied over and that the TLS certs are still working.
  • Remove /etc/hosts hacks from all mirrors.

Post migration steps
  • Tweet and dent about the fact that the migration was successful.
  • Send a test email to the support address included in the tweet/dent.
  • Take a backup of config files and data on the old server in case I forgot to copy something to the new one.
  • Get in touch with mirror owners to tell them to update libravatar-slave package and test ssh configuration.
  • Add third-party controlled mirrors back to the DNS load-balancer once they are up to date.
  • A few days later, change the TTL for the main site back to 43200 seconds.
  • A week later, kill the proxy on the old server by shutting it down.

4 August 2013

Francois Marier: Debugging Gearman configuration

Gearman is a queuing system that has been in Debian for a long time and is quite reliable. I ran into problems however when upgrading a server from Debian squeeze to wheezy however. Here's how I debugged my Gearman setup.

Log verbosity First of all, I started by increasing the verbosity level of the daemon by adding --verbose=INFO to /etc/default/gearman-job-server (the possible values of the verbose option are in the libgearman documentation) and restarting the daemon:
/etc/init.d/gearman-job-server restart
I opened a second terminal to keep an eye on the logs:
tail -f /var/log/gearman-job-server/gearman.log

Listing available workers Next, I registered a very simple worker:
gearman -w -f mytest cat
and made sure it was connected properly by telneting into the Gearman process:
telnet localhost 4730
and listing all currently connected workers using the workers command (one of the commands available in the Gearman TEXT protocol). There should be an entry similar to this one:
30 127.0.0.1 - : mytest
Because there are no exit or quit commands in the TEXT protocol, you need to terminate the telnet connection like this:
  1. Press Return
  2. Press Ctrl + ]
  3. Press Return
  4. Type quit at the telnet prompt and press Return.
Finally, I sent some input to the simple worker I setup earlier:
echo "hi there"   gearman -f mytest
and got my input repeated on the terminal:
hi there

Gearman bug I traced my problems down to this error message when I sent input to the worker:
gearman: gearman_client_run_tasks : connect_poll(Connection refused)
getsockopt() failed -> libgearman/connection.cc:104
It turns out that it is a known bug that was fixed upstream but still affects Debian wheezy and some versions of Ubuntu. The bug report is pretty unhelpful since the work-around is hidden away in the comments of this "invalid" answer: be explicit about the hostname and port number in both gearman calls. So I was able to make it work like this:
gearman -w -h 127.0.0.1 -p 4730 -f mytest cat    
echo "hi there"   gearman -h 127.0.0.1 -p 4730 -f mytest
where the hostname matches exactly what's in /etc/default/gearman-job-server.

28 July 2013

Francois Marier: FISL for foreigners HOWTO

FISL (pronounced FIZZ lay by the locals) is a large Free and Open Source software gathering in Porto Alegre, Brazil. While the primary audience of that conference is the Latin America "libre software" community, a number of overseas speakers also participate in that conference. This is my contribution to them: a short guide for international guests visiting the F rum Internacional do Software Livre.

Planning Before you fly out, make sure you look up the visa requirements for your country of citizenship. Many western countries will require a visa and you will need to visit the local Brazillian embassy ahead of time to get one. Next, have a look at the list of recommended immunizations. As with most destinations, it is recommended that your routine immunizations be up to date, but there are also other specialized ones such as Yellow Fever that are recommended by the Brazillian government. You should therefore visit a travel clinic a few weeks ahead of time. Other than that, I suggest reading up on the country and keeping an eye on the various travel advisories.

Arrival You will be flying at the Porto Alegre airport. If you need to exchange overseas money for Brazilian Reals, you can do that there. You'll probably also want to pick up a power adapter at the airport if you intend to charge your laptop while you're in the country :) Brazil has both 127V and 220V outlets using Type N sockets. Privacy note: using the free airport wifi will require giving your Passport details as part of the registration process.

Language If you don't speak Portuguese, expect a few challenges since most of the people you'll meet (including taxi drivers, many airport workers, some hotel staff) won't speak English. I highly recommend getting a phrase book before you leave and printing paper maps of where you are planning to go (to show to taxi drivers when you get lost). Native Spanish speakers seem to get by speaking Spanish to Portuguese speakers and understanding enough Portuguese to hold a conversation. I wouldn't count on it unless your Spanish is quite good though. Also, the official conference blog posts get eventually translated to English, but there is a delay, so you may want to subscribe to the Portuguese feed and use Google Translate to keep up with FISL news before you get there.

The conference FISL is a large conference and it has a very "decentralized" feel to it. From the outside, it looks like it's organized by an army of volunteers where everyone is taking care of some portion of it without a whole lot of top-down direction. It seems to work quite well! What this means for you as a foreign speaker however is that you're unlikely to be provided with a lot of information or help finding your way around the conference (i.e. no "speaker liaison"). There is a separate registration desk for speakers but that's about all of the attention you'll receive before you deliver your talk. So make sure you know where to go and show up in your assigned room early to speak with the person introducing you. If your talk is in English, it will be live-transated by an interpreter. It's therefore a good idea to speak a bit more slowly and to pause a bit more. Other than that, the organizers make an effort to schedule an English talk in each timeslot so non-Portuguese speakers should still be able to get a lot out of the conference. FISL was a lot of fun for me and I hope that some of these tips will help you enjoy the biggest FLOSS gathering in the southern hemisphere!

18 May 2013

Francois Marier: Three wrappers to run commands without impacting the rest of the system

Most UNIX users have heard of the nice utility used to run a command with a lower priority to make sure that it only runs when nothing more important is trying to get a hold of the CPU:
nice long_running_script.sh
That's only dealing with part of the problem though because the CPU is not all there is. A low priority command could still be interfering with other tasks by stealing valuable I/O cycles (e.g. accessing the hard drive).

Prioritizing I/O Another Linux command, ionice, allows users to set the I/O priority to be lower than all other processes. Here's how to make sure that a script doesn't get to do any I/O unless the resource it wants to use is idle:
sudo ionice -c3 hammer_disk.sh
The above only works as root, but the following is a pretty good approximation that works for non-root users as well:
ionice -n7 hammer_disk.sh
You may think that running a command with both nice and ionice would have absolutely no impact on other tasks running on the same machine, but there is one more aspect to consider, at least on machines with limited memory: the disk cache.

Polluting the disk cache If you run a command (for example a program that goes through the entire file system checking various things, you will find that the kernel will start pulling more files into its cache and expunge cache entries used by other processes. This can have a very significant impact on a system as useful portions of memory are swapped out. For example, on my laptop, the nightly debsums, rkhunter and tiger cron jobs essentially clear my disk cache of useful entries and force the system to slowly page everything back into memory as I unlock my screen saver in the morning. Thankfully, there is now a solution for this in Debian: the nocache package. This is what my long-running cron jobs now look like:
nocache ionice -c3 nice long_running.sh

Turning off disk syncs Another relatively unknown tool, which I would certainly not recommend for all cron jobs but is nevertheless related to I/O, is eatmydata. If you wrap it around a command, it will run without bothering to periodically make sure that it flushes any changes to disk. This can speed things up significantly but it should obviously not be used for anything that has important side effects or that cannot be re-run in case of failure. After all, its name is very appropriate. It will eat your data!

1 April 2013

Francois Marier: Poor man's RAID1 between an SSD and a hard drive

After moving from a hard drive to an SSD on my work laptop, I decided to keep the hard drive spinning and use it as a backup for the SSD. With the following setup, I can pull the SSD out of my laptop and it should still boot up normally with all of my data on the hard drive.

Manually setting up an encrypted root partition Before setting up the synchronization between the two drives, I had to replicate the partition setup. I used fdisk, cfdisk and gparted to create the following partitions:
   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048      499711      248832   83  Linux
/dev/sdb2          501760   500117503   249807872    5  Extended
/dev/sdb5          503808   500117503   249806848   83  Linux
and then loosely followed these instructions to create an encrypted root partition on /dev/sdb5:
$ cryptsetup luksFormat /dev/sdb5
$ cryptsetup luksOpen /dev/sdb5 sdb5_crypt
$ pvcreate /dev/mapper/sdb5_crypt
$ vgcreate akranes2 /dev/mapper/sdb5_crypt
$ vgchange -a y akranes2
$ lvcreate -L247329718272B -nroot akranes2
$ lvcreate -L8468299776B -nswap_1 akranes2
$ mkfs.ext4 /dev/akranes2/root
Finally, I added the new encrypted partition to the list of drives to bring up at boot time by looking up its UUID:
$ cryptsetup luksUUID sdb5_crypt
and creating a new entry for it in /etc/crypttab.

Copying the boot partition Setting up the boot partition was much easier because it's not encrypted. All that was needed was to format it and then copy the files over:
$ mkfs.ext2 /dev/sdb1
$ mount /dev/sdb1 /mnt/boot
$ cp -a /boot/* /mnt/boot/
The only other thing to remember is to install grub on the boot loader of that drive. On modern Debian systems, that's usually just a matter of running dpkg-reconfigure grub-pc and adding the second drive (/dev/sdb in my case) to the list of drives to install grub on.

Sync scripts To keep the contents of the SSD and the hard drive in sync, I set up a regular rsync of the root and boot partitions using the following mount points (as defined in /etc/fstab):
/dev/mapper/akranes-root /           ext4    noatime,discard,errors=remount-ro 0       1
/dev/mapper/akranes2-root /mnt/root  ext4    noatime,errors=remount-ro         0       2
UUID=0b9109d0-... /boot              ext2    defaults                          0       2
UUID=6e6f05fb-... /mnt/boot          ext2    defaults                          0       2
I use this script (/usr/local/sbin/ssd_boot_backup) for syncing the boot partition:
#!/bin/sh
if [ ! -e /mnt/boot/hdd.mounted ] ; then
    echo "The rotating hard drive is not mounted in /mnt/boot."
    exit 1
fi
if [ ! -e /boot/ssd.mounted ] ; then
    echo "The ssd is not the boot partition"
    exit 1
fi
nice ionice -c3 rsync -aHx --delete --exclude=/ssd.mounted --exclude=/lost+found/* /boot/* /mnt/boot/
and a similar one (/usr/local/sbin/ssd_root_backup) for the root partition:
#!/bin/sh
if [ ! -e /mnt/root/hdd.mounted ] ; then
    echo "The rotating hard drive is not mounted in /mnt/root."
    exit 1
fi
if [ ! -e /ssd.mounted ] ; then
    echo "The ssd is not the root partition"
    exit 1
fi
nice ionice -c3 rsync -aHx --delete --exclude=/dev/* --exclude=/proc/* --exclude=/sys/* --exclude=/tmp/* --exclude=/boot/* --exclude=/mnt/* --exclude=/lost+found/* --exclude=/media/* --exclude=/var/tmp/* --exclude=/ssd.mounted --exclude=/var/lib/lightdm/.gvfs --exclude=/home/francois/.gvfs /* /mnt/root/
To ensure that each drive is properly mounted before the scripts run, I created empty ssd.mounted files in the root directory of each of the partitions on the SSD, and empty hdd.mounted files in the root directory of the hard drive partitions.

Cron jobs The sync scripts are run every couple of hours through this crontab:
10 */4 * * *                root    /usr/local/sbin/ssd_boot_backup
20 0,4,8,12,16,20 * * *     root    /usr/local/sbin/ssd_root_backup
20 2,6,10,14,18,22 * * *    root    /usr/bin/on_ac_power && /usr/local/sbin/ssd_root_backup
which includes a reduced frequency while running on battery to avoid spinning the hard drive up too much.

22 February 2013

Francois Marier: Doing a fresh Debian/Ubuntu install without having to reconfigure everything

Taking advantage of a new hard drive, I decided to reinstall my Ubuntu (Precise 12.04.2) laptop from scratch so that I could easily enable full-disk encryption (a requirement for Mozilla work laptops). Reinstalling and reconfiguring everything takes a bit of time though, so here's the procedure I followed to keep the configuration to a minimum.

Install Ubuntu/Debian on the new drive While full-disk encryption is built into the graphical installer as of Ubuntu 12.10, it's only available through the alternate install CD in 12.04 and earlier. Using that CD, install Ubuntu on the new drive, making sure you select the "Encrypted LVM" option when you get to the partitioning step. (The procedure is the same if you use a Debian CD.) To make things easy, the first user you create should match exactly the one from your previous installation.

Copy your home directory Once the OS is installed on the new drive, plug the old one back in (in an external enclosure if you need to) and mount its root partition as a read-only drive on /mnt. Then log in as root (not just using sudo, actually login using the root user) and copy your home directory on the new drive:
rm -rf /home/*
cp -a /mnt/home/* /home/
Then you should be able to log in as your regular user.

Reinstall all packages The next step is to reinstall all of the packages you had installed on the old OS. But first of all, let's avoid having to answer all of the debconf questions we've already answered in the past:
rm -rf /var/cache/debconf
cp -a /mnt/var/cache/debconf /var/cache/
and set the debconf priority to the one you usually use (medium in my case):
dpkg-reconfigure debconf
Next, make sure you have access to all of the necessary repositories:
cp -a /mnt/etc/apt/sources.list /etc/apt/
cp -a /mnt/etc/apt/sources.list.d/* /etc/apt/sources.list.d/
apt-get update

Getting and setting the list of installed packages To get a list of the packages that were installed on the old drive, use dpkg on the old install:
mount -o remount,rw /mnt
chroot /mnt
dpkg --get-selections > /packages
exit
mount -o remount,ro -r /mnt
Use that list on the new install to reinstall everything:
dpkg --set-selections < /mnt/packages
apt-get dselect-upgrade

Selectively copy configuration files over Finally, once all packages are installed, you can selectively copy the config files from the old drive (in /mnt/etc) to the new (/etc/). In particular, make sure you include these ones:
cp -a /mnt/etc/alternatives/ /mnt/etc/default/ /etc/
(I chose not to just overwrite all config files with the old ones because I wanted to get rid of any cruft that had accumulated there and suspected that there might be some slight differences due to the fresh install of the distro.)

Reboot Once that's done, you should really give that box a restart to ensure that all service are using the right config files, but otherwise, that's it. Your personal data and config are back, all of your packages are installed and configured the way they were, and everything is fully encrypted!

5 February 2013

Joey Hess: LCA2013 wrapup

After 27 hours of travel, I'm finally back from my first Linux.conf.au. This was a great conference! Some of my favorite highlights:
My git-annex talk went ok, despite technical problems with the video output, which the video team did a great job of working around. The first demo went well, and the Q&A was excellent. The demo of the git-annex assistant webapp was marred by, apparently, a bad build of the webapp -- which I ironically used rather than my usual development build because I assumed it'd be less likely to fail. At least I may have a way to reproduce that hang somewhat reliably now, so I can get on with debugging it. I will be redoing a demo of the webapp as a screencast.
Here are some of the best talks I attended. (I have quite a lot more queued up to watch still, and had to cut several due to space.) Click titles for videos, or browse all the videos.

16 January 2013

Francois Marier: Moving from Blogger to Ikiwiki and Branchable

In order to move my blog to a free-as-in-freedom platform and support the great work that Joey (of git-annex fame) and Lars (of GTD for hackers fame) have put into their service, I decided to convert my Blogger blog to Ikiwiki and host it on Branchable. While the Ikiwiki tips page points to some old instructions, they weren't particularly useful to me. Here are the steps I followed.

Exporting posts and comments from Blogger Thanks to Google letting people export their own data from their services, I was able to get a full dump (posts, comments and metadata) of my blog in Atom format. To do this, go into "Settings Other" then look under "Blog tools" for the "Export blog" link.

Converting HTML posts to Markdown Converting posts from HTML to Markdown involved a few steps:
  1. Converting the post content using a small conversion library to which I added a few hacks.
  2. Creating the file hierarchy that ikiwiki requires.
  3. Downloading images from Blogger and fixing their paths in the article text.
  4. Extracting comments and linking them to the right posts.
The Python script I wrote to do all of the above will hopefully be a good starting point for anybody wanting to migrate to Ikiwiki.

Maintaining old URLs In order to make sure I wouldn't break any existing links pointing to my blog on Blogger, I got the above Python script to output a list of Apache redirect rules and then found out that I could simply email these rules to Joey and Lars to get them added to my blog. My rules look like this:
# Tagged feeds
Redirect permanent /feeds/posts/default/-/debian http://feeding.cloud.geek.nz/tags/debian/index.rss
Redirect permanent /search/label/debian http://feeding.cloud.geek.nz/tags/debian
# Main feed (needs to come after the tagged feeds)
Redirect permanent /feeds/posts/default http://feeding.cloud.geek.nz/index.rss
# Articles
Redirect permanent /2012/12/keeping-gmail-in-separate-browser.html http://feeding.cloud.geek.nz/posts/keeping-gmail-in-separate-browser/
Redirect permanent /2012/11/prefetching-resources-to-prime-browser.html http://feeding.cloud.geek.nz/posts/prefetching-resources-to-prime-browser/

Collecting analytics Since I am no longer using Google Analytics on my blog, I decided to take advantage of the access log download feature that Joey recently added to Branchable. Every night, I download my blog's access log and then process it using awstats. Here is the cron job I use:
#!/bin/bash
BASEDIR=/home/francois/documents/branchable-logs
LOGDIR=/var/log/feedingthecloud
# Download the current access log
LANG=C LC_PAPER= ssh -oIdentityFile=$BASEDIR/branchable-logbot b-feedingthecloud@feedingthecloud.branchable.com logdump > $LOGDIR/access.log
It uses a separate SSH key I added through the Branchable control panel and outputs to a file that gets overwritten every day. Next, I installed the awstats Debian package, and configured it like this:
$ cat /etc/awstats/awstats.conf.local
SiteDomain=feedingthecloud.branchable.com
LogType=W
LogFormat=1
LogFile="/var/log/feedingthecloud/access.log"
Even if you're not interested in analytics, I recommend you keep an eye on the 404 errors for a little while after the move. This has helped me catch a critical redirection I had forgotten.

Limiting Planet feeds One of the most common things that happen right after someone migrates to a new blogging platform is the flooding of any aggregator that subscribes to their blog. The usual cause being the change in post identifiers. Unsurprisingly, Ikiwiki already had a few ways to avoid this problem. I chose to simply modify each tagged feed and limit them to the posts added after the move to Branchable.

Switching DNS Having always hosted my blog on a domain I own, all I needed to do to move over to the new platform without an outage was to change my CNAME to point to feedingthecloud.branchable.com. I've kept the Blogger blog alive and listening on feeding.cloud.geek.nz to ensure that clients using a broken DNS resolver (which caches records for longer than requested via the record's TTL) continue to see the old posts.

20 December 2012

Francois Marier: Keeping GMail in a separate browser profile

I wanted to be able to use the GMail web interface on my work machine, but for privacy reasons, I prefer not to be logged into my Google Account on my main browser. Here's how I make use of a somewhat hidden Firefox feature to move GMail to a separate browser profile. Creating a separate profile The idea behing browser profiles is simple: each profile has separate history, settings, bookmarks, cookies, etc. To create a new one, simply start Firefox with this option:
firefox -ProfileManager
to display a dialog which allows you to create new profiles:
Once you've created a new "GMail" profile, you can start it up from the profile manager or directly from the command-line:
firefox -no-remote -P GMail
(The -no-remote option ensures that a new browser process is created for it.) To make this easier, I put the command above in a tiny gmail shell script that lives in my ~/bin/ directory. I can use it to start my "GMail browser" by simply typing gmail. Tuning privacy settings for 2-step authentication While I initially kept that browser profile in private browsing mode, this was forcing me to enter my 2-factor authentication credentials every time I started the browser. So to avoid having to use Google Authenticator (or its Firefox OS cousin) every day, I ended up switching to custom privacy settings and enabling all cookies:
It turns out however that there is a Firefox extension which can selectively delete unwanted cookies while keeping useful ones. Once that add-on is installed and the browser restarted, simply add accounts.google.com to the whitelist and set it to clear cookies when the browser is closed:
Then log into GMail and tick the "Trust this computer" checkbox at the 2-factor prompt:
With these settings, your browsing history will be cleared and you will be logged out of GMail every time you close your browser but will still be able to skip the 2-factor step on that device.

26 October 2012

Dirk Eddelbuettel: Accelerating R code: Computing Implied Volatilities Orders of Magnitude Faster

This blog, together with Romain's, is one of the main homes of stories about how Rcpp can help with getting code to run faster in the context of the R system for statistical programming and analysis. By making it easier to get already existing C or C++ code to R, or equally to extend R with new C++ code, Rcpp can help in getting stuff done. And it is often fairly straightforward to do so. In this context, I have a nice new example. And for once, it is work-related. I generally cannot share too much of what we do there as this is, well, proprietary, but I have this nice new example. The other day, I was constructing (large) time series of implied volatilities. Implied volatilities can be thought of as the complement to an option's price: given a price (and all other observables which can be thought of as fixed), we compute an implied volatility price (typically via the standard Black-Scholes model). Given a changed implied volatility, we infer a new price -- see this Wikipedia page for more details. In essence, it opens the door to all sorts of arbitrage and relative value pricing adventures. Now, we observe prices fairly frequently to create somewhat sizeable time series of option prices. And each price corresponds to one matching implied volatility, and for each such price we have to solve a small and straightforward optimization problem: to compute the implied volatility given the price. This is usually done with an iterative root finder. The problem comes from the fact that we have to do this (i) over and over and over for large data sets, and (ii) that there are a number of callbacks from the (generic) solver to the (standard) option pricer. So our first approach was to just call the corresponding function GBSVolatility from the fOption package from the trusted Rmetrics project by Diethelm Wuertz et al. This worked fine, but even with the usual tricks of splitting over multiple cores/machines, it simply took too long for the resolution and data amount we desired. One of the problems is that this function (which uses the proper uniroot optimizer in R) is not inefficient per se, but simply makes to many function call back to the option pricer as can be seen from a quick glance at the code. The helper function .fGBSVolatility gets called time and time again:
R> GBSVolatility
function (price, TypeFlag = c("c", "p"), S, X, Time, r, b, tol = .Machine$double.eps, 
    maxiter = 10000) 
 
    TypeFlag = TypeFlag[1]
    volatility = uniroot(.fGBSVolatility, interval = c(-10, 10), 
        price = price, TypeFlag = TypeFlag, S = S, X = X, Time = Time, 
        r = r, b = b, tol = tol, maxiter = maxiter)$root
    volatility
 
<environment: namespace:fOptions>
R> 
R> .fGBSVolatility
function (x, price, TypeFlag, S, X, Time, r, b, ...) 
 
    GBS = GBSOption(TypeFlag = TypeFlag, S = S, X = X, Time = Time, 
        r = r, b = b, sigma = x)@price
    price - GBS
 
<environment: namespace:fOptions>
So the next idea was to try the corresponding function from my RQuantLib package which brings (parts of) QuantLib to R. That was seen as been lots faster already. Now, QuantLib is pretty big and so is RQuantLib, and we felt it may not make sense to install it on a number of machines just for this simple problem. So one evening this week I noodled around for an hour or two and combined (i) a basic Black/Scholes calculation and (ii) a standard univariate zero finder (both of which can be found or described in numerous places) to minimize the difference between the observed price and the price given an implied volatility. With about one hundred lines in C++, I had something which felt fast enough. So today I hooked this into R via a two-line wrapper in quickly-created package using Rcpp. I had one more advantage here. For our time series problem, the majority of the parameters (strike, time to maturity, rate, ...) are fixed, so we can structure the problem to be vectorised right from the start. I cannot share the code or more the details of my new implementation. However, both GBSVolatility and EuropeanOprionImpliedVolatility are on CRAN (and as I happen to maintain these for Debian, also just one sudo apt-get install r-cran-foptions r-cran-rquantlib away if you're on Debian or Ubuntu). And writing the other solver is really not that involved. Anyway, here is the result, courtesy of a quick run via the rbenchmark package. We create a vector of length 500; the implied volatility computation will be performed at each point (and yes, our time series are much longer indeed). This is replicated 100 times (as is the default for rbenchmark) for each of the three approaches:
xyz@xxxxxxxx:~$ r xxxxR/packages/xxxxOptions/demo/timing.R
    test replications elapsed  relative user.self sys.self user.child sys.child
3 zzz(X)          100   0.038     1.000     0.040    0.000          0         0
2 RQL(X)          100   3.657    96.237     3.596    0.060          0         0
1 fOp(X)          100 448.060 11791.053   446.644    1.436          0         0
xyz@xxxxxxxx:~$ 
The new local solution is denoted by zzz(X). It is already orders of magnitude faster than the RQL(x) function using RQuantLib (which is, I presume, due to my custom solution internalising the loop). And the new approach is a laughable amount faster than the basic approach (shown as fOp) via fOptions. For one hundred replications of solving implied volatilities for all elements of a vector of size 500, the slow solution takes about 7.5 minutes --- while the fast solution takes 38 milliseconds. Which comes to a relative gain of over 11,000. So sitting down with your C++ compiler to craft a quick one-hundred lines, combining two well-known and tested methods, can reap sizeable benefits. And Rcpp makes it trivial to call this from R.

18 September 2012

Francois Marier: Advice to newcomers and veterans of the Free and Open Source software communities

A few months ago, a collection of essays called Open Advice was released. It consists of the things that seasoned Free and Open Source developers wish they had known when they got started. The LWN book review is the best way to get a good feel for what it's about, but here are the notes I took while reading it:

15 July 2012

Francois Marier: Browsing privacy and ad blocking on Android

On the desktop, I usually rely on Privoxy to strip out ads, tracking resources and other privacy-invading elements. So I was looking for an equivalent solution on Android. Firefox 10 With the current version of Firefox for Android, you can simply install the Adblock Plus extension and it will filter most undesirable elements from webpages. Unfortunately, that extension is not yet available for the latest Firefox Beta, so I had to find another solution. Android Adblock It turns out that there is an Open Source proxy similar to Privoxy (though much more limited in functionality) available for Android: Adblock (also available on the F-Droid Free Software market). However, its default configuration really doesn't block much and so you'll probably want to import a new blocklist as soon as you install it. I used a combination of the Easylist and EasyPrivacy blocklists. Configuring Fennec to use a proxy Unlike its desktop cousin, Firefox for Android (also called Fennec) doesn't expose proxy settings in the user interface. Instead, you have to open the about:config page and configure the following settings manually:

network.proxy.http = localhost
network.proxy.http_port = 8080
network.proxy.ssl = localhost
network.proxy.ssl_port = 8080
network.proxy.type = 1
Once you're done, test your connection by going into the AdBlock application and turning the proxy off. Then switch back to Firefox and go to a new website. You should get an error message telling you that the proxy is blocking connections. That means it's successfully using your proxy to talk to other websites and not connecting to them directly. (It might also be possible to set this up in the default Android browser or in the Chrome for Android Beta, but I haven't been able to find how. Feel free to leave a comment if you know how it's done.) Bonus tips While you're at it, I highly recommend you turn on the Do Not Track feature in Firefox. Some large sites (like Twitter) have recently committed to turning off individual tracking on web requests which contain this new privacy header. Also, if you want to help move the mobile web away from a WebKit monoculture (remember how bad the Internet Explorer 6 monoculture was for the web?), then please consider joining the Mobile Testdrivers team and help us make Firefox rock on Android!

14 July 2012

Romain Francoise: Getting clickable URLs in xterm (sort of)

My terminal emulator of choice is xterm: it's fast, light, and (since it's pretty much the reference implementation) it has excellent support for everything a console user could desire... except for one thing that would be very, very convenient: making URLs clickable. Other terminal emulators have this feature, but they also have problems that make them inferior to xterm in different ways. Over the years people have come up with various workarounds for this situation, like screen scraping utilities (urlview, urlscan) that can be hooked up to other console programs to extract and browse URLs, but they're still not as convenient as just using the mouse, and often require the program to run on the same machine as the browser. Fortunately, xterm #277 (released in January 2012) added a new feature that provides almost exactly what I had been looking for: it can now spawn programs using the exec-formatted action and give them as argument the contents of the current selection or clipboard. So you can add the following to your ~/.Xresources: *VT100*translations: #override Meta <Btn1Up>: exec-formatted("x-www-browser '%t'", PRIMARY) which makes xterm run x-www-browser on the selection when it receives Alt + left click. (Adjust for whatever your Meta key is.) This is advantageously combined with a charClass setting to make xterm treat URLs as a single word, so that you can just double-click on them to select them: XTerm*charClass: 33:48,36-47:48,58-59:48,61:48,63-64:48,95:48,126:48 With both of these enabled, opening URLs is now just a matter of:
  1. Double-clicking the URL to select it
  2. Doing Alt + click anywhere on the xterm window to run the browser
which, while more involved than a single click, it still much faster than having to copy the URL manually to the browser.

1 June 2012

Francois Marier: Proper indentation of Javascript files using js2-mode in emacs

If you use emacs for Javascript or Node.js development, you should have a look at js2-mode (apt-get install js2-mode in Debian / Ubuntu). In addition to providing syntax highlight, it will parse your Javascript and issue errors and warnings. (It's not as pedantic as jslint or jshint so you'll probably want to fix all of them before committing your files.) Unfortunately the default indentation style looks a bit crazy and adds excessive indentation to your code. Here's an example:

navigator.id.secret.generate(identity, function (plainKey, wrappedKey)
var data = wrappedKey: wrappedKey ;
$.post('/loggedin', data, function (res)
$("#message").text('All done');
, 'json');
, function (error)
$("#message").text('ERROR: ' + error);
);
It turns out that indenting Javascript properly is really hard, but you can turn on a special flag which will cause the indentation to "bounce" between different strategies, starting with the most likely one. This is what I now have in my ~/.emacs:
(custom-set-variables
'(js2-basic-offset 2)
'(js2-bounce-indent-p t)
)
You can find and configure all other options by issuing the following command:
M-x customize-group [RET] js2-mode [RET]
So yes, you can have both the reasonable indentation of the standard js-mode and the helpful warnings and errors of js2-mode!

24 March 2012

Dirk Eddelbuettel: Initial release 0.1.0 of package RcppSMC

Hm, I realized that I announced this on Google+ (via Rcpp) as well as on Twitter, on the r-packages list, wrote a new and simple web page for it, but had not put it on my blog. So here is some catching up. Sequential Monte Carlo / Particle Filter is a (to quote the Wikipedia page I just linked to) sophisticated model estimation technique based on simulation. They are related to both Kalman Filters, and Markov Chain Monte Carlo methods. Adam Johansen has a rather nice set of C++ classes documentated in his 2009 paper in the Journal of Statistical Software (JSS). I started to play with these classes and realized that, once again, this would make perfect sense in an R extension built with the Rcpp package by Romain and myself (and in JSS too). So I put a first prototype onto R-Forge and emailed Adam who, to my pleasant surprise, was quite interested. And a couple of emails, and commits later, we are happy to present a very first release 0.1.0. I wrote a few words on a RcppSMC page on my website where you can find a few more details. But in short, we already have example functions demonstrating the backend classes by reproducing examples from
Johansen (2009)
and his example 5.1 via pfLineartBS() for a linear bootstrap example;
Doucet, Briers and Senecal (2006)
and their (optimal) block-sampling particle filter for a linear Gaussian model (serving as an illustration as the setup does of course have an analytical solution) via the function blockpfGaussianOpt()
Gordon, Salmond and Smith (1993)
and their ubiqitous nonlinear state space model via the function pfNonlinBS().
And to illustrate just why Rcpp is so cool for this, here is a little animation of a callback from the C++ code when doing the filtering on Adam's example 5.1. By passing a simple plotting function, written in R, to the C++ code, we can get a plot updated on every iteration. Here I cheated a little and used our old plot function with fixed ranges, the package now uses a more general function: Example of RcppSMC callback to R plot when estimation example 5.1 from Johansen (2009) The animation is of course due to ImageMagick glueing one hundred files into a single animated gif. More information about RcppSMC is on its page, and we intend to add more examples and extensions over time.

12 March 2012

Francois Marier: Creating a FreeDOS bootable USB stick to upgrade BIOS

I have an old motherboard that requires creating a DOS boot floppy in order to upgrade its BIOS. Fortunately, it's not too hard to do this with FreeDOS and a USB stick.

The instructions below are based on an FDos wiki article.

Downloading the dependenciesThe first step is to download the required files from your motherboard manufacturer:
and then install the tools you'll need:
apt-get install makebootfat syslinux

Preparing the "floppy" imageStart by collecting all of the files you need to install FreeDOS on the USB stick:
cd /tmp

wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/commandx.zip
wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/kernels.zip
wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/substx.zip
wget http://www.ibiblio.org/pub/micro/pc-stuff/freedos/files/distributions/1.0/pkgs/unstablx.zip

for ZIP in *.zip; do unzip $ZIP; done

cp ./source/ukernel/boot/fat16.bin .
cp ./source/ukernel/boot/fat12.bin .
cp ./source/ukernel/boot/fat32lba.bin .

cp /usr/lib/syslinux/mbr.bin .
and then create a directory for the files that will end up in the root directory of the "floppy":
mkdir /tmp/fs-root
cp ./bin/command.com /tmp/fs-root/
cp ./bin/kernel.sys /tmp/fs-root/
and copy the BIOS image and update program into that same directory (/tmp/fs-root/).

Creating a bootable USB stickPlug in a FAT-formatted USB stick and look for the device it uses (/dev/sdb in the example below).

Finally, run makebootfat:
/usr/bin/makebootfat -o /dev/sdb -E 255 -1 fat12.bin -2 fat16.bin -3 fat32lba.bin -m mbr.bin /tmp/fs-root

Next.

Previous.